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ABSTRACT 

Instructional product development-- the purpose of 
which has been to create materials which produce in an identified 
population of users demonstrable changes in behavior, in accordanc 
with prespecified and specific objectives— generally adheres to a 
pattern which includes planning, formulating, prototyping measures 
and materials, field testing, revising and retesting, and summative 
evaluation. To the project staff at the Institute for Development of 
Educational Activities, Inc. (IDEA) , this cycle seemed worthwhile to 
explore in terms of producing materials which would enable groups to 
improve interpersonal problem-solving skills. Thus, IDEA developed 
materials designed to help teachers and principal work together more 
collaboratively anvd systematically in solving school problems. Field 
tests of the materials indicated the need for revisions, and 
evaluations of the program are currently formative. However, the use 
of product development strategies for organizational development has 
proven to be of value in serving as a research tool, in forcing 
clarification of desired outcomes and principles used to attain them, 
and in expanding our knowledge base with its empirical approach. 
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Summary: Recent emphasis on insuring the quality of instructional materials for 

. students in terms of predictable learning outcomes has resulted in attention to system- 
atic procedures for developing such materials, A modification of these systematic 
procedures was used by the staff of the Institute for Development of Educational Activities 
Inc. (|i|d|e|a|) to develop materials for teachers to influence the way in which they 
’ worked together to solve problems. This paper describes the process by which these 
materials were developed and discusses some of the implications which product develop- 
ment strategies may have for the field of organizational development. 



Introduction 

Within the past ten years increasing attention has been paid to a sub -speciality 

I 

within the field of education called instructional product development. Interest in in- 

I 

structionai product development was stimulated by the many curriculum projects during ^ 

the sixties. Some of these projects tried to translate current knowledge in disciplines 
such as mathematics, science, economics, history as well as scholarly methodology 
into materials which would be effective in the classroom. Further impetus for materials 
development has come from federally funded regional laboratories which are presently f 

K. 

trying to package for school people - whether stud'^^'.ts, teachers or administrators - re- 
search based knowledge in a maximally useful form. 

The purpose of product development efforts has been to create materials which 
produce in an identified population of users demonstrable changes in behavior, in accord- 

* Paper presented at the 1972 conference of the American Educational Research 
Association in Chicago, Illinois. 
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ance with prespecified and specific objectives. In order to do this, product developers 

incorporate into their first draft materials instructional principles which research has 

* 

shown to have a high probability of facilitating learning. However, because there is no 
conclusive way to predict the effects of these principles in actual practice, product 
developers then test their materials with representative samples of users, and revise 
them in accordance with this empirical data. Product developers, through successive 
trial and revision cycles, try to produce instructional artifacts - books, films, tapes, 
exercises - which take responsibility for learner achievement of given objectives. 

To date, most product development efforts have been concerned with cognitive 
outcomes for individuals or collections of indi^^iduals. Current development strategy 
reflects this emphasis on individual .learning. 

The Product Development Cycle 

Although product development agencies differ in the way they conceptualize the 
stages of development, most adliere to a pattern which includes planning, formulating, 
prototyping measures and materials, field testing, revising and retesting, and summative 
evaluation. ^ 

During the planning stage, the need for the product is ascertained through review 
of presently available materials and surveys of potential consumers as well as of experts 
in the field. During the formulation stage, the broad areas of content are defined and 
within those areas the most economical concepts are identified. Specifications describing 
the objectives of the product as well as the measures for determining if the objectives 
liave been achieved are written. The population for whom the product is intended is 
described and the entry skills which learners must already posess in order to begin 



using the product are identified. 



3 



After the formulation stage,, prototype materials, procedures and measures 
are developed according to specification and tried out with small samples of learners. 

Based on data from this initial tryout, materials and measures are revised and tried 
out again, with larger groups of learners and under conditions more nearly approximate 
ing those under which the product will be used. Supporting conditions essential to the 
Installation of the product are considered. Adjunct materials such as instructor’s 
manuals are developed; or necessary arrangements such as release time for teacher 
training are made. Additional revisions are made if necessary and the product is 
ready to be distributed. Finally, summative data describing the functioning of the 
product in its intended setting and the learner outcomes associated with the use of the 
product are collected, and made available so that potential users can decide if the pro- 
duct is appropriate for their needs. The defining characteristic of the product develop- ^ 

f 

ment approach, then, is its emphasis on learner achievement as indicated by empirical 

I 

evidence. ^ 

X. 

This development cycle of sequential steps for producing materials which enable ^ 

individual learners to acquire cognitive skills seemed worthwhile to explore in terms of 
producing materials which would enable groups of people to improve interpersonal problem 
solving skills. The project staff at the Institute for Development of Educational Activities, r 

Inc. (|i|d|e|a|) used these procedures or a modification of them to create a set of mate- 
rials which would help school staffs become more effective in solving problems. 

Developing the Problem Solving School Materials 
For several years prior to the beginning of this project, members of the |i|d|e|a| 
staff had been engaged in a study of change in eighteen elementary schools in Southern 
California joined together in a League of Cooperating Schools. The tentative results of 
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this study supports other research 2 which indicates that it is not usual practice for 
teachers within a school to collaboratively and systematically engage in dialogue 
about either school-level problems or classroom-level problems. The assumption 
underlying the work of the League and of this project was that if school staffs could 
become better able to collaborate with one another and to systematically identify, 
analyze and generate alternative solutions to problems, the educational function of 
the school would be performed more effectively. 

The limited number of consultants available to the schools and the widespread 
interest in schools in staff development activities seemed to indicate that an exploration 
of the way in which products could bring about changes in the behavior of school staffs 
was justified. 

In formulating the product, then, we at |i|d|e|a| agreed- that the appropriate 
audience for the materials was the entire staff of the elementary school. The general 
pr:‘pose of the materials was to enable teachers and principal to work together more 
collaboratively and more systematically in solving school level and class level problems. 
It became clear, however, in identifying the content for the materials that there were 
differences as well as similarities in school and classroom "problem solving; and so two 
sets of materials were designed to be used either concurrently or consecutively. * 



*The materials for school level problem solving are described in a handbook 
distributed to teachers, principal, and to members of the Development Team who 
had been selected by the staff to coordinate the activities of the program. The hand- 
book includes guidelines for three meetings at which team-building activities are con- 
ducted (some of these are guided by taped instructions). The staff then engages in 
seven additional activities for problem identification, diagnosis, development of al- 
ternative solutions, evaluation oi alternatives, implementation and evaluation of group 
effectiveness. The classroom problem solving materials are contained in a set of 
nine programed booklets which deal with problem definition, information collection, 
alternative strategies and evaluation. They are to be read individually, then discussed 
in a small group of teachers with similar problems. Suggestions are made for practice 

in the classroom. 
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The principles used in the development of the materials were derived both 
from learning theory and organizational development theory. For instance, I'ne 
materials call for active participation by the entire staff in the program both during 
meetings and between meetings. Some activities were designed for total staff meet- 
ings, some for subgroups within a total staff meeting, others for individual 
use. Various techniques for participant monitoring of activities and for feeding back 
th3 results of that monitoring to the group are provided. There are numerous oppor- 
cunities for graduated practice built into the products. For instance, the classroom 
booklets ask teachers to respond in writing to hypothetical situations in terms of the 
principles introduced in the booklets, then to respond in writing to a situation within 
dieir own classroom, then to discuss their responses with a small group of teachers, 
and finally to practice such a procedure within their own classroom. 

During the fomulation stage of product development, we found it difficult to 
proceed in the systematic mamier we had planned. The two areas which caused the 
most trouble were the definition of school entry behaviors deemed prerequisite for 
using the program, and the specification of group outcomes in measurable terms. 
Although we. were aware that products by themselves were not powerful change agents, 
and that their success depended upon the existence of certain pre-conditions such as 
adequate motivation on the part of the staff, willing leadership on the part of the prin- 
cipal, and the capabilities of the individuals selected as Development Team members, 
we were unable at that time to devise instruments to differentiate between schools 
likely to succeed with the materials and those likely to fail. We did insist that prin- 
cipals involve their staffs in the decision about participating in the [i|d|e|a| study; 
and this turned out to be a significant variable. Those schools in which teachers did 
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actively participate in tlie decision to become involved with |i|d|e|a| , or where 
there was an initial misunderstanding about the nature of the program either did not 
start, or dropped out early in the program. 

The difficulty with specifying behavioral outcomes for groups using the materials 
can be traced to two factors. First, there was genuine reluctance on the part of many 
members of the project staff to prescribe outcomes at all. Specifying objectives in 
precise and measurable language seemed to imply that we were imposing value judg- 
ments on school staffs by designing exercises and materials which were closed ended. 

Second, staff members perceived difficulties in establishing a logical relationship 
between a specific skill objective such as ’’teachers will be able to engage in brain- 
storming activities to generate several alternative solutions to a problem” and the 
larger general objectives of improvement in collaborative and systematic problem 
solving activities. 

I 

We therefore postponed the specification of behavioral outcomes until after the 

V 

development of the materials and the first field test stage. * Instead of specifying 
outcomes, we selected activities which, in the literature, seemed to help people 

^Samples of current behavioral outcomes include: 

f 

There will be an increase in openness, trust, and participatory decision-making 
among staff members (indicated by self-report questionnaires). 

Staff members will be able to use the following systematic procedures in reaching 
a solution to a problem: force -field analysis, brainstorming, evaluating alternatives, 
choosing a course of action, implementing a decision (indicated by observation data). 

Teachers will be able to describe a given classroom problem in terms of the discrep- 
ancy between teacher expectations and pupil behaviors (indicated by paper and pencil 
items as well as reports of small group discussion). 

Teachers will be able to use the strategies of establishing no-lose relationship, rein- 
forcement, adapting the task, changing the context, in their classrooms (as indicated 
by paper and pencil items as well as self reports). 
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acquire the kinds of behaviors we thought would be useful. We then packaged these 
activities in the form of guidelines for meetings, or in the form of exercises, and 
tried them out in schools. Based on observation by staff members and on self-report 
questionnaires filled out by teachers, we tried to eliminate the ’’noise” and refine 
the procedures so that they would efficiently affect people’s behaviors. 

The first field test of the product was conducted activity by activity. The full 
Staff team building and problem solving exercises were introduced into schools one 
at a time. Evaluations by participants as well as observations by staff members 
resulted in extensive revision. The classroom level booklets were read by individual 
teachers and their reactions were carefully noted by the developer who sat with them 

q 

in a "counselling interview. ’’ The booklets then were distributed in schools, and 
read by teachers at their own rate. Posttest data as well as evaluative judgment 
data led to revisions. 

Tlie entire program has been effect in five schools during the current school 

year, and 'there is continous monitoring of school progress by the jljDjEjAj staff. 

This year’s evaluation is still formative in that it will lead to further revisions in 

* 

form, content, sequence, instructions, etc. * 

Summative evaluation concerning the effect of the entire program on collaborative 
and systematic problem solving abilities of school staffs will have to await another 
year of testing. Although it is still too early to reach conclusions about the effective- 
ness of this product in bringing about staff development in public school settings, 

* A more detailed description of the formative evaluation procedures can be 
found in two papers presented at the California Educational Research Association. 

San Diego, California, November, 1971, "The Problem Solving School: Product Develop- 
ment and Formative Evaluation" by Adrianne Bank and "Team Building Activities: Purposes, 
Description and School Reactions" by Roger Rasmussen. 



some implications of the product development process for organizational develop- 
ment can be noted. 

Implications of Product Development Strategies for 
Organizational Development 

There are obviously serious limitations to the product as change agent. They 
are inherent in the medium itself which cannot adapt to the .specific needs of organ- 
izational systems in the flexible and creative ways that consultants can. Materials 
are limited to their capacity to sustain motivation in groups of people, and in their 
power to reveal to people the complexities of their interpersonal functioning. Materials 
cannot take the place of leadership. 

Products, however, have advantages for organizational development in at least 
two areas. They can serve as an important research tool. Because they are replicable, 
and can be used in exactly the same form in a variety of situations, they can help to 
identify important differences among organizational settings. Because selected variables 
such as method of presentation, time of presentation, type of instructions can be manipul- 
ated under controlled conditions, more precise information about the effects of specific 
interventions in organization! settings can be obtained. Investigation concerning the 
limits of such techniques as videotape analysis and feedback, or of group process ex- 
ercises can be made, so that conclusions can be reached concerning the most appropriate 
use of limited consultant resources. The possible dangers of do-it-yourself organizational 
development efforts can be more clearly analyzed and described. 

The product development process itself forces clarification of the desired outcomes 
and the principles used to achieve them. Although it may be appropriate to delay or 
avoid altogether specifying behavioral outcomes for an organizational development effort. 
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and although it may be undesirable or impossible to construct measures for ascertain- 
ing the long range effectiveness of intervention activities, the demands of the product 
development cycle that such decisions be considered is salutary. In a time when the 
supply of knowledgeable organizational development consultants is limited and the 
demand is great, it seems inevitable that replicable materials will be developed to 
extend their expertise.. These materials can build upon and expand the knowledge 
base ^vithin the field. If emphasis is correctly placed on empirical development and 
testing procedures and if this data is made available to consumer .schools, informed 
decisions can be made by school staffs about the direction in which they want to proceed. 

Much time, energy and manpower is required to explore the ways in which products 
and product development can contribute to the theory and practice of organizational 
development. We at jljDjEjAj know this only too well. 
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1. For more extensive treatment of the product development cycle, see Popham, 
W. James and Baker, Eva L. ’’Rules for the Development of Instructional 
Products. ” Southwest Regional Laboratory for Educational Research and 
Development, 1967. 

2. Goodlad, John I. and Klein, Frances M. Behind the Classroom Door , 
Worthington, Ohio. Charles A. Jones, 1970. 

3. Markle, Susan. ’’Empirical Testing of Programs” Programed Instruction, 
The sixty -sixth yearbook of the National Society for the Stu^ of Education, 
Part II, University of Chicago Press, 1967, p. 104-138. 
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